1. Field of the Invention
The present invention relates to an apparatus that performs an interpolation process in order to interpolate a pixel value for a pixel having a missing pixel value or an incorrect pixel value in an image, and a pixel interpolation method performed by the device.
2. Description of the Related Art
A scanner device that scans images using an optical unit is classified into a device that employs a contact image sensor (CIS) and a device that employs a charge coupled device (CCD). The scanner device employing the CIS is unable to scan a stereoscopic document since the document needs to be in close contact with a surface to be scanned. However, in recent years, the device employing the CIS has been in wide use due to its thin body and an inexpensive cost as compared to the device employing the CCD as well as an improvement in the technique of reducing noise that is likely to occur in a scanned image. For example, the CIS is used in an image scanner having a one-pass duplex scanning function in order to scan the rear surface of a document.
The CIS uses RGB light emitting diodes (LEDs) as a light source, emits R, G, and B color lights to a document by switching them at a high speed, and inputs the reflected lights from the document to an imaging element (CMOS image sensor) through a lens. The CMOS image sensor converts the lights input from the document into voltage values pixel by pixel and outputs the converted voltage values.
The scanner device employing the CIS employs a contact image sensor type in which a document is brought into close contact with a sensor using a roller and the document is scanned line by line. It is difficult to manufacture a single long CIS, and thus, a scanning unit of the CIS is formed by arranging a plurality of short sensors is arranged in their longitudinal directions. However, in this configuration, since a certain gap is formed between the sensors, an image signal in this gap may not be acquired. As a result, a deficiency in the image signal occurs.
In a scanner device, the image signal may become deficient or the scanned image signal may have an incorrect value due to a defect of a sensor scanning the image signal or due to the presence of an obstacle in the midway of an optical path such as dust present on a contact glass on which a document is set.
As a result, there is a problem in that a pixel having a missing pixel value or an incorrect pixel value is present in a scanned image, and the image quality deteriorates. To solve this problem, in the related art, there is known a method of estimating a correct pixel value of a target pixel having a missing or incorrect pixel value from pixel values of the surrounding pixels thereof and interpolating (inserting) the estimated pixel value instead of the missing or incorrect pixel value of the target pixel.
For example, a method of performing linear interpolation using the pixel values of the surrounding pixels of a target pixel; and a method of performing polynomial interpolation or spline interpolation using a function of the second order or higher are known. The interpolation method based on linear interpolation is ideal for interpolation at portions of an image having a small change in density but is not ideal for interpolation at portions of an image having a severe change in density like a halftone dot region.
The interpolating method based on polynomial interpolation and spline interpolation can estimate pixel values with high accuracy when the sampling cycle for sampling an image such as a digital photograph is sufficiently shorter than the variation cycle of the image pattern. However, in the case of halftone dot images, since the image resolution is not sufficient as compared to the number of lines of halftone dots, the sampling period is not sufficient in relation to the variation period of the image. Thus, the interpolating method may be unable to reproduce the original pattern correctly.
Thus, in order to solve the problem of the interpolating method, a method which uses pattern matching has been proposed. In this pattern matching method, it is expected that high-frequency components which could not be reproduced by the interpolating method can be reproduced using similar patterns in the vicinity of the interpolation target pixel.
However, in general, since the pattern matching uses a large range of information as compared to the interpolating method, although the optimum solution to a base pattern is obtained, the obtained solution is not said to be optimum to the interpolation target pixel. This problem happens because when searching for similar patterns in the pattern matching method, patterns which are slightly different from each other as a whole are not distinguished from patterns which are mostly identical but partly greatly different from each other.
In particular, like a halftone dot region having a high screen ruling, when information is eccentrically distributed to specific pixels, a method of selecting similar patterns has a large effect on the interpolation result. Moreover, in a halftone dot region having a low density, since the proportion of pixels belonging to the background region of an image is relatively high, similar patterns may be detected in the background region rather than the halftone dot region, or alternatively, no similar pattern may be detected.
Therefore, when performing interpolation, it is necessary to use an interpolation method depending on the characteristics of a region to which the interpolation target pixel belongs. In order to accurately estimate the pixel value of the interpolation target pixel, there has been proposed a device in which when the position of an interpolation target pixel is within a halftone dot region, patterns similar to a pattern including the interpolation target pixel are searched for from an image, and the pixel value of a pixel, which is included in the most similar pattern and corresponds to the interpolation target pixel, is determined as the pixel value of the interpolation target pixel (see Japanese Patent No. 4330164).
This device determines whether the position of an interpolation target pixel corresponding to a connecting portion of contact image sensors is within a halftone dot region or a non-halftone dot region based on the image data scanned by an image scanning sensor. When the position of the interpolation target pixel is within the non-halftone dot region, the pixel data of the interpolation target pixel are generated by means of linear interpolation, and the generated pixel data is inserted into the interpolation target pixel. On the other hand, when the position of the interpolation target pixel is within the halftone dot region, the pixel data of the interpolation target pixel are generated by means of pattern matching, and the generated pixel data is inserted into the interpolation target pixel.
In this case, one base block including the interpolation target pixel and a plurality of reference blocks having the same size as the base block but not including the interpolation target pixel are set in an image region near the position of the interpolation target pixel. Then, the correlation values between the base block and the respective reference blocks are calculated based on the pixel data within the base block and the pixel data within the respective reference blocks. Among the pixel data within a reference block having the highest correlation value with the base block, the pixel data of a pixel corresponding to the interpolation target pixel within the base block are determined as the pixel data of the interpolation target pixel.
In this interpolation method, the pixel value of the interpolation target pixel can be determined using similar patterns present around the interpolation target pixel. However, since the size of a template used in pattern matching and the range for searching for the similar patterns thereof are constant regardless of an image, there is a problem in that the interpolation accuracy decreases for patterns having a short cycle such as a halftone dot having a high screen ruling.
When a template of a halftone-dot image having a high screen ruling is set to have a fixed size, a plurality of halftone dots are included in the template. If similar patterns are searched for using that template, patterns having a high degree of coincidence with a plurality of halftone dots included in the template are selected. However, this method does not always result in the optimum result for a halftone dot including the interpolation target pixel. As a result, an interpolation result which is away from the optimum value is obtained, and image quality may decrease.
Therefore, there is a demand for a device and a method capable of dynamically determining the template size and the range to be searched to retrieve the similar patterns in accordance with the characteristics of the image and of performing an interpolation process with high accuracy with respect to patterns having a short cycle such as a halftone dot having a high screen ruling.